Discrete-time Birth-death Processes Research Articles

A Review of Birth-Death and Other Markovian Discrete-Time Queues

In this review article, we consider discrete-time birth-death processes and their applications to discrete-time queues. To make the analysis simpler to follow, we focus on transform-free methods and consider instances of non-birth-death Markovian discrete-time systems. We present a number of results within one discrete-time framework that parallels the treatment of continuous time models. This approach has two advantages; first, it unifies the treatment of several discrete-time models in one framework, and second, it parallels to the extent possible the treatment of continuous time models. This allows us to draw parallels and contrasts between the discrete and continuous time queues. Specifically, we focus on birth-death applications to the single server discrete-time model with Bernoulli arrivals and geometric service times and provide the reader with a simple rigorous detailed analysis that covers all five scheduling rules considered in the literature, with attention to stationary distributions at slot edges, slot centers, and prearrival epochs. We also cover the waiting time distributions. Moreover, we cover three Markovian models that fit the global balance equations. Our approach provides interesting insights into the behavior of discrete-time queues. The article is intended for those who are familiar with queueing theory basics and would like a simple, yet rigorous introductory treatment to discrete-time queues.

On time-discretized versions of the stochastic SIS epidemic model: a comparative analysis.

In this paper, the interest is in the use of time-discretized models as approximations to the continuous-time birth-death (BD) process [Formula: see text] describing the number I(t) of infective hosts at time t in the stochastic [Formula: see text] (SIS) epidemic model under the assumption of an additional source of infection from the environment. We illustrate some simple techniques for analyzing discrete-time versions of the continuous-time BD process [Formula: see text], and we show the similarities and differences between the discrete-time BD process [Formula: see text] of Allen and Burgin (Math Biosci 163:1-33, 2000), which is inspired from the infinitesimal transition probabilities of [Formula: see text], and an alternative discrete-time Markov chain [Formula: see text], which is defined in terms of the number [Formula: see text] of infective hosts at a sequence [Formula: see text] of inspection times. Processes [Formula: see text] and [Formula: see text] can be thought of as a uniformized version and the discrete skeleton of process [Formula: see text], respectively, and are commonly used to derive, in the more general setting of Markov chains, theorems about a continuous-time Markov chain by applying known theorems for discrete-time Markov chains. We shall demonstrate here that the continuous-time BD process [Formula: see text] and its discrete-time counterparts [Formula: see text] and [Formula: see text] behave asymptotically the same in the limit of large time index, while the processes [Formula: see text] and [Formula: see text] differ from the continuous-time BD process [Formula: see text] in terms of the random length of an outbreak, or when considering their dynamics during a predetermined time interval [Formula: see text]. To compare the dynamics of process [Formula: see text] with those of the discrete-time processes [Formula: see text] and [Formula: see text] during [Formula: see text], we consider extreme values (i.e., maximum and minimum number of infectives simultaneously observed during [Formula: see text]) in these three processes. Finally, we illustrate our analytical results by means of a number of numerical examples, where we use the Hellinger distance between two probability distributions to quantify the similarity between the resulting extreme value distributions of either [Formula: see text] and [Formula: see text], or [Formula: see text] and [Formula: see text].

On a property of random walk polynomials involving Christoffel functions

Discrete-time birth-death processes may or may not have certain properties known as asymptotic aperiodicity and the strong ratio limit property. In all cases known to us a suitably normalized process having one property also possesses the other, suggesting equivalence of the two properties for a normalized process. We show that equivalence may be translated into a property involving Christoffel functions for a type of orthogonal polynomials known as random walk polynomials. The prevalence of this property – and thus the equivalence of asymptotic aperiodicity and the strong ratio limit property for a normalized birth-death process – is proven under mild regularity conditions.

On the Strong Ratio Limit Property for Discrete-Time Birth-Death Processes

A sufficient condition is obtained for a discrete-time birth-death process to possess the strong ratio limit property, directly in terms of the one-step transition probabilities of the process. The condition encompasses all previously known sufficient conditions.

Counterintuitive properties of the fixation time in network-structured populations.

Evolutionary dynamics on graphs can lead to many interesting and counterintuitive findings. We study the Moran process, a discrete time birth–death process, that describes the invasion of a mutant type into a population of wild-type individuals. Remarkably, the fixation probability of a single mutant is the same on all regular networks. But non-regular networks can increase or decrease the fixation probability. While the time until fixation formally depends on the same transition probabilities as the fixation probabilities, there is no obvious relation between them. For example, an amplifier of selection, which increases the fixation probability and thus decreases the number of mutations needed until one of them is successful, can at the same time slow down the process of fixation. Based on small networks, we show analytically that (i) the time to fixation can decrease when links are removed from the network and (ii) the node providing the best starting conditions in terms of the shortest fixation time depends on the fitness of the mutant. Our results are obtained analytically on small networks, but numerical simulations show that they are qualitatively valid even in much larger populations.

The generalized time variable reconstructed birth–death process

Much recent research has investigated the effect that different time variable birth–death processes have on the distribution of branching times in phylogenies of extant taxa. Previous work has shown how to calculate the distributions of number of lineages and branching times for a reconstructed constant rate birth–death process that started with one or two reconstructed lineages at some time in the past or ended with some number of lineages in the present. Here I expand that work to include any time variable birth–death process that starts with any number of reconstructed lineages and/or ends with any number of reconstructed lineages at any time, and I calculate a number of distributions under that process. I also explore the discrete time birth–death process which operates as an efficient and accurate numerical solution to any time-variable birth–death process and allows for the analytical incorporation of sampling and mass extinctions. I describe how these distributions can be used to compare different time variable models using maximum likelihood analysis, and I show how to simulate random trees under any of these models. I also introduce two visual methods for evaluating different time variable birth–death processes; these methods illustrate the shape of distributions for the number of lineages and waiting times by plotting them over time.

Additive Functionals for Discrete-Time Markov Chains with Applications to Birth-Death Processes

In this paper we are interested in bounding or calculating the additive functionals of the first return time on a set for discrete-time Markov chains on a countable state space, which is motivated by investigating ergodic theory and central limit theorems. To do so, we introduce the theory of the minimal nonnegative solution. This theory combined with some other techniques is proved useful for investigating the additive functionals. This method is used to study the functionals for discrete-time birth-death processes, and the polynomial convergence and a central limit theorem are derived.

Additive Functionals for Discrete-Time Markov Chains with Applications to Birth-Death Processes

In this paper we are interested in bounding or calculating the additive functionals of the first return time on a set for discrete-time Markov chains on a countable state space, which is motivated by investigating ergodic theory and central limit theorems. To do so, we introduce the theory of the minimal nonnegative solution. This theory combined with some other techniques is proved useful for investigating the additive functionals. This method is used to study the functionals for discrete-time birth-death processes, and the polynomial convergence and a central limit theorem are derived.

Convergence rates for reversible Markov chains without the assumption of nonnegative definite matrices

Explicit convergence rates in geometric and strong ergodicity for denumerable discrete time Markov chains with general reversible transition matrices are obtained in terms of the geometric moments or uniform moments of the hitting times to a fixed point. Another way by Lyapunov’s drift conditions is also used to derive these convergence rates. As a typical example, the discrete time birth-death process (random walk) is studied and the explicit criteria for geometric ergodicity are presented.

On mixing rate and convegence to stationary regime in discrete time Erlang problem

Sufficient conditions for polynomial convergence rate to the stationary regime and beta-mixing for some classes of ergodic discrete time birth-death processes are established.

Further approaches to computing fundamental characteristics of birth-death processes

General and unifying approaches are discussed for computing fundamental characteristics of both continuous-time and discrete-time birth-death processes. In particular, an exponential family framework is used to derive explicit expressions, in terms of continued fractions, for joint generating functions of first-passage times and a whole collection of associated random quantities, and a random sum representation is used to obtain formulae for means, variances and covariances of stopped reward functions defined on a birth-death process.

Further approaches to computing fundamental characteristics of birth-death processes

General and unifying approaches are discussed for computing fundamental characteristics of both continuous-time and discrete-time birth-death processes. In particular, an exponential family framework is used to derive explicit expressions, in terms of continued fractions, for joint generating functions of first-passage times and a whole collection of associated random quantities, and a random sum representation is used to obtain formulae for means, variances and covariances of stopped reward functions defined on a birth-death process.

Analysis of random walks using orthogonal polynomials

We discuss some aspects of discrete-time birth-death processes or random walks, highlighting the role played by orthogonal polynomials.

Weak convergence of conditioned birth-death processes in discrete time

We consider a discrete-time birth-death process on the non-negative integers with −1 as an absorbing state and study the limiting behaviour as n → ∞ of the process conditioned on non-absorption until time n. By proving that a condition recently proposed by Martinez and Vares is vacuously true, we establish that the conditioned process is always weakly convergent when all self-transition probabilities are zero. In the aperiodic case we obtain a necessary and sufficient condition for weak convergence.

Weak convergence of conditioned birth-death processes in discrete time

We consider a discrete-time birth-death process on the non-negative integers with −1 as an absorbing state and study the limiting behaviour asn →∞ of the process conditioned on non-absorption until timen.By proving that a condition recently proposed by Martinez and Vares is vacuously true, we establish that the conditioned process is always weakly convergent when all self-transition probabilities are zero. In the aperiodic case we obtain a necessary and sufficient condition for weak convergence.

Geomatric ergodicity and quasi-stationarity in discrete-time birth-death processes

AbstractWe study two aspects of discrete-time birth-death processes, the common feature of which is the central role played by the decay parameter of the process. First, conditions for geometric ergodicity and bounds for the decay parameter are obtained. Then the existence and structure of quasi-stationary distributions are discussed. The analyses are based on the spectral representation for the n-step transition probabilities of a birth-death process developed by Karlin and McGregor.

Ratio Limits and Limiting Conditional Distributions for Discrete-Time Birth-Death Processes

We consider discrete-time birth-death processes with an absorbing state and study the conditional state distribution at time n given that absorption has not occurred by that time but will occur eventually. In particular, we establish conditions for the convergence of these distributions to a proper distribution as n→∞. The problem turns out to be closely related to that of finding conditions for the existence of limits of ratios of n-step transition probabilities as n→∞. Orthogonal polynomials feature in the spectral representation for the n-step transition probabilities of a birth-death process and, consequently, play a key role in the analysis.